Real-time acquisition of depth and color images using structured light and its application to 3D face recognition

In this paper, a novel real-time 3D and color sensor for the mid-distance range (0.1-3m) based on color-encoded structured light is presented. The sensor is integrated using lowcost of-the-shelf components and allows the combination of 2D and 3D image processing algorithms, since it provides a 2D color image of the scene in addition to the range data. Its design is focused on enabling the system to operate reliably in real-world scenarios, i.e. in uncontrolled environments and with arbitrary scenes. To that end, novel approaches for encoding and recognizing the projected light are used, which make the system practically independent of intrinsic object colors and minimize the influence of the ambient light conditions. The system was designed to assist and complement a face authentication system integrating both 2D and 3D images. Depth information is used for robust face detection, localization and 3D pose estimation. To cope with illumination and pose variations, 3D Preprint submitted to Real-Time Imaging 17 June 2005 information is used for the normalization of the input images. The performance and robustness of the proposed system is tested on a face database recorded in conditions similar to those encountered in real-world applications.